Crowdbreaks: Tracking Health Trends using Public Social Media Data and Crowdsourcing

In the past decade, tracking health trends using social media data has shown great promise, due to a powerful combination of massive adoption of social media around the world, and increasingly potent hardware and software that enables us to work with these new big data streams. At the same time, many challenging problems have been identified. First, there is often a mismatch between how rapidly online data can change, and how rapidly algorithms are updated, which means that there is limited reusability for algorithms trained on past data as their performance decreases over time. Second, much of the work is focusing on specific issues during a specific past period in time, even though public health institutions would need flexible tools to assess multiple evolving situations in real time. Third, most tools providing such capabilities are proprietary systems with little algorithmic or data transparency, and thus little buy-in from the global public health and research community. Here, we introduce Crowdbreaks, an open platform which allows tracking of health trends by making use of continuous crowdsourced labelling of public social media content. The system is built in a way which automatizes the typical workflow from data collection, filtering, labelling and training of machine learning classifiers and therefore can greatly accelerate the research process in the public health domain. This work introduces the technical aspects of the platform and explores its future use cases

Crowdbreaks: Tracking Health Trends Using Public Social Media Data and Crowdsourcing

In the past decade, tracking health trends using social media data has shown great promise, due to a powerful combination of massive adoption of social media around the world, and increasingly potent hardware and software that enables us to work with these new big data streams. At the same time, many challenging problems have been identified. First, there is often a mismatch between how rapidly online data can change, and how rapidly algorithms are updated, which means that there is limited reusability for algorithms trained on past data as their performance decreases over time. Second, much of the work is focusing on specific issues during a specific past period in time, even though public health institutions would need flexible tools to assess multiple evolving situations in real time. Third, most tools providing such capabilities are proprietary systems with little algorithmic or data transparency, and thus little buy-in from the global public health and research community. Here, we introduce Crowdbreaks, an open platform which allows tracking of health trends by making use of continuous crowdsourced labeling of public social media content. The system is built in a way which automatizes the typical workflow from data collection, filtering, labeling and training of machine learning classifiers and therefore can greatly accelerate the research process in the public health domain. This work describes the technical aspects of the platform, thereby covering the functionalities at its current state and exploring its future use cases and extensions

On the Ground Validation of Online Diagnosis with Twitter and Medical Records

Social media has been considered as a data source for tracking disease. However, most analyses are based on models that prioritize strong correlation with population-level disease rates over determining whether or not specific individual users are actually sick. Taking a different approach, we develop a novel system for social-media based disease detection at the individual level using a sample of professionally diagnosed individuals. Specifically, we develop a system for making an accurate influenza diagnosis based on an individual's publicly available Twitter data. We find that about half (17/35 = 48.57%) of the users in our sample that were sick explicitly discuss their disease on Twitter. By developing a meta classifier that combines text analysis, anomaly detection, and social network analysis, we are able to diagnose an individual with greater than 99% accuracy even if she does not discuss her health.Comment: Presented at of WWW2014. WWW'14 Companion, April 7-11, 2014, Seoul, Kore

On the Ground Validation of Online Diagnosis with Twitter and Medical Records

Social media has been considered as a data source for tracking disease. However, most analyses are based on models that prioritize strong correlation with population-level disease rates over determining whether or not specific individual users are actually sick. Taking a different approach, we develop a novel system for social-media based disease detection at the individual level using a sample of professionally diagnosed individuals. Specifically, we develop a system for making an accurate influenza diagnosis based on an individual's publicly available Twitter data. We find that about half (17/35 = 48.57%) of the users in our sample that were sick explicitly discuss their disease on Twitter. By developing a meta classifier that combines text analysis, anomaly detection, and social network analysis, we are able to diagnose an individual with greater than 99% accuracy even if she does not discuss her health.Comment: Presented at of WWW2014. WWW'14 Companion, April 7-11, 2014, Seoul, Kore

The Red Queen and the persistence of linkage-disequilibrium oscillations in finite and infinite populations

<p>Abstract</p> <p>Background</p> <p>The Red Queen Hypothesis (RQH) suggests that the coevolutionary dynamics of host-parasite systems can generate selection for increased host recombination. Since host-parasite interactions often have a strong genetic basis, recombination between different hosts can increase the fraction of novel and potentially resistant offspring genotypes. A prerequisite for this mechanism is that host-parasite interactions generate persistent oscillations of linkage disequilibria (LD).</p> <p>Results</p> <p>We use deterministic and stochastic models to investigate the persistence of LD oscillations and its impact on the RQH. The standard models of the Red Queen dynamics exhibit persistent LD oscillations under most circumstances. Here, we show that altering the standard model from discrete to continuous time or from simultaneous to sequential updating results in damped LD oscillations. This suggests that LD oscillations are structurally not robust. We then show that in a stochastic regime, drift can counteract this dampening and maintain the oscillations. In addition, we show that the amplitude of the oscillations and therefore the strength of the resulting selection for or against recombination are inversely proportional to the size of the (host) population.</p> <p>Conclusion</p> <p>We find that host parasite-interactions cannot generally maintain oscillations in the absence of drift. As a consequence, the RQH can strongly depend on population size and should therefore not be interpreted as a purely deterministic hypothesis.</p